Real physical objects interacting with augmented reality features

ABSTRACT

Embodiments herein provide a method for determining, utilizing an output from a beacon device detected using one or more sensors, a physical location of an interactive device in a physical space. Additionally, the method includes receiving an indication that an augmented reality scene is being displayed, where the augmented reality scene includes the physical space and a first virtual element. The method also includes identifying a predefined dynamic based on characteristics of the interactive device and the first virtual element. Finally, the method includes determining a physical movement to perform based on the determined physical location of the interactive device and the predefined dynamic, and activating the one or more actuators to cause the determined physical movement.

BACKGROUND Field of the Invention

The present disclosure generally relates to augmented reality, and morespecifically, to providing immersive interaction between physicalobjects and virtual elements in augmented reality scenes.

Description of the Related Art

Virtual reality (VR) and augmented reality (AR) have become increasinglypopular in recent years. Virtual reality generally involves simulating avirtual environment such that the user feels as if he or she is actuallypresent in the virtual world. With a typical VR environment, the actualphysical space around the user is not visible or displayed to the user,and only virtual elements are displayed. Augmented reality involvesdisplaying at least a portion of the real physical environment aroundthe user, with virtual elements augmenting the physical scene, such asby projecting the virtual elements into the physical environment.Similarly, the virtual elements can be projected or displayed on ascreen that is partially transparent so that the real physical world isstill visible, or one or more cameras can capture the physicalenvironment and display it to the user, along with the virtual elements.

AR allows the user to see virtual objects and elements as if they arepresent in the physical environment. For example, to the user, it mayappear that a virtual character is standing in the user's room.Unfortunately, it is not possible for real objects to interact withvirtual stimuli. For example, physical objects such as balls and actionfigures cannot react to the virtual elements being displayed to theuser. There is a need to provide for immersive interaction betweenphysical and virtual objects and features.

SUMMARY

One embodiment disclosed herein is a method for determining, utilizingan output from a beacon device detected using one or more sensors, aphysical location of an interactive device in a physical space.Additionally, the method includes receiving an indication that anaugmented reality scene is being displayed, where the augmented realityscene includes the physical space and a first virtual element. Themethod also includes identifying a predefined dynamic based oncharacteristics of the interactive device and the first virtual element.Finally, the method includes determining a physical movement to performbased on the determined physical location of the interactive device andthe predefined dynamic, and activating the one or more actuators tocause the determined physical movement.

Another embodiment disclosed herein is an interactive device comprisinga processor, one or more sensor devices, one or more actuators, anetwork interface, and a memory containing computer program code that,when executed on the processor, performs an operation. The operationcomprises determining, utilizing an output from a beacon device detectedusing one or more sensors, a physical location of the interactive devicein a physical space. Additionally, the operation includes receiving anindication that an augmented reality scene is being displayed, where theaugmented reality scene includes the physical space and a first virtualelement. The operation also includes identifying a predefined dynamicbased on characteristics of the interactive device and the first virtualelement. Finally, the operation includes determining a physical movementto perform based on the determined physical location of the interactivedevice and the predefined dynamic, and activating the one or moreactuators to cause the determined physical movement.

Another embodiment disclosed herein is a system including a beacondevice and an augmented reality device including a first processor and afirst memory containing first computer program code that, when executedon the first processor, performs a first operation. The system alsoincludes an interactive device including a second processor, one or moreactuators, and a second memory containing second computer program codethat, when executed on the second processor, performs a secondoperation. The first operation includes determining, using detectedoutput from the beacon device, a physical location of the augmentedreality device in a physical space, and displaying an augmented realityscene, where the augmented reality scene includes the physical space anda first virtual element. The second operation includes determining,using detected output from the beacon device, a physical location of theinteractive device in the physical space. The second operation alsoincludes identifying a predefined dynamic based on characteristics ofthe interactive device and the first virtual element. Further, thesecond operation includes determining a physical movement to performbased on the determined physical location of the interactive device andthe predefined dynamic, and activating the one or more actuators tocause the determined physical movement.

BRIEF DESCRIPTION OF THE DRAWINGS

So that the manner in which the above recited aspects are attained andcan be understood in detail, a more particular description ofembodiments of the invention, briefly summarized above, may be had byreference to the appended drawings.

It is to be noted, however, that the appended drawings illustrate onlytypical embodiments of this invention and are therefore not to beconsidered limiting of its scope, for the invention may admit to otherequally effective embodiments.

FIG. 1 illustrates a system including an augmented reality device, abeacon device, and an interactive device, according to one embodimentdisclosed herein.

FIG. 2 is a block diagram of an augmented reality device and aninteractive device, according to one embodiment disclosed herein.

FIGS. 3A, 3B, and 3C illustrate an augmented reality scene, according toone embodiment of the present disclosure.

FIGS. 4A, 4B, and 4C illustrate an augmented reality scene, according toone embodiment of the present disclosure.

FIG. 5 is a flow chart illustrating a workflow for providing interactionbetween physical devices and virtual elements, according to anembodiment disclosed herein.

FIG. 6 is a flow chart illustrating a workflow for providing interactionbetween physical devices and virtual elements, according to anembodiment disclosed herein.

FIG. 7 is a flow chart illustrating a method of providing interactivephysical devices, according to one embodiment of the present disclosure.

FIG. 8 is a flow chart illustrating a method of providing interactivephysical devices, according to one embodiment of the present disclosure.

DETAILED DESCRIPTION

Augmented reality scenes are unique in that they include both thephysical world around the user, as well as elements and features of avirtual environment, superimposed into the physical environment. In thisway, the user can observe the virtual elements as if they exist in thephysical world. For example, virtual characters can appear to stand inthe room the user is in.

In order to interact with virtual features, interactive devices mustknow the position of the virtual element (e.g., its location andorientation) relative to itself. Additionally, in order to interact withthe user, the interactive device must know the position of the user.Existing approaches to augmented reality include display devices thatcan track virtual elements, but other physical devices do not know theirown position, nor do they know the position of the virtual elements. Assuch, true interaction between physical devices and virtual elements isnot possible with existing solutions.

In an embodiment, interactive physical devices can determine their ownlocation in a physical scene. Similarly, the augmented reality device(e.g., a head-mounted display that displays the augmented scene)determines its location in the environment. In one embodiment of thepresent disclosure, the determined location of the augmented realitydevice and the interactive physical device can be used along with thelocation of the virtual elements to cause the interactive device and thevirtual element to react to each other and provide a more immersive andengaging experience.

FIG. 1 illustrates a System 100 including an Augmented Reality Device105, a Beacon Device 120, and an Interactive Device 125, according toone embodiment disclosed herein. As illustrated, the Augmented RealityDevice 105 is a head-mounted display that the user wears using straps115. Additionally, the Augmented Reality Device 105 includes two Sensors110. In an embodiment, the Sensors 110 are cameras or other imagingsensors. In other embodiments, the Sensors 110 may be other types ofsensor, such as ultrasonic sensors, infrared sensors, and the like. Inembodiments, the Sensors 110 are utilized by the Augmented RealityDevice 105 to determine its location in the physical environment.

Although the illustrated embodiment includes two Sensors 110, in someembodiments, the Augmented Reality Device 105 includes a single Sensor110, or more than two Sensors 110. Similarly, although both Sensors 110are illustrated as facing the same direction, in some embodiments theSensors 110 may face in different directions from each other. In anembodiment, each Sensor 110 is a different type of sensor. Further, insome embodiments, additional sensors may be included in or on theAugmented Reality Device 105, such as accelerometers, tilt sensors,magnetometers, and the like.

In embodiments, the Augmented Reality Device 105 displays elements ofvirtual scenes (e.g., virtual objects, features, characters, and thelike) to the user such that they appear to be present in the physicalspace around the user. For example, in one embodiment, the AugmentedReality Device 105 includes a screen or display, and one or more of theSensors 110 is a camera. In such an embodiment, the screen may displaythe images retrieved by the Sensor 110 (e.g., images of the physicalenvironment around the user), with the virtual elements superimposed onthem.

In some embodiments, the Augmented Reality Device 105 includes asemi-transparent screen such that the physical world can be seen throughthe screen, and the virtual elements can be displayed or projected onit. In some embodiments, the screen is a half-silvered mirror or beamsplitter that allows the virtual elements to be reflected off the screenand to the user's eyes, while allowing the user to view the physicalenvironment as well. In an embodiment, a mobile device such as asmartphone is used as the Augmented Reality Device 105, and is placed inor on a headset or other device so that the display and various sensorsof the mobile device can be utilized. Generally, any method may be usedto display the augmented scene, where the augmented scene includes boththe physical space around the user and one or more virtual elements.

In an embodiment, a Beacon Device 120 can be used to help locate theAugmented Reality Device 105 in the physical space. In an embodiment,the Beacon Device 120 projects light (which may be in the visiblespectrum, infrared, radio waves, and the like) that can be detected bythe Sensors 110 of the Augmented Reality Device 105. For example, in theillustrated embodiment, the Augmented Reality Device 105 utilizes theSensors 110 to detect the Beacon Device 120, as illustrated by lines135. In an embodiment, based on the detected light or signals from theBeacon Device 120, the Augmented Reality Device 105 can determine itslocation in the physical space relative to the Beacon Device 120.

For example, in an embodiment, the perceived location of the BeaconDevice 120 to the Sensors 110 can be used to determine the directionand/or angle that the Augmented Reality Device 105 is facing at anygiven point in time, relative to the Beacon Device 120. Similarly, in anembodiment, the Beacon Device 120 can output different light or signalsit on different sides or faces, such that the Augmented Reality Device105 can determine where it is in relation to the Beacon Device 120 basedon the light that the Sensors 110 detect. In an embodiment, the detectedintensity of the Beacon Device 120 can be used to determine the distancethe Augmented Reality Device 105 is from the Beacon Device 120.

In an embodiment where Augmented Reality Device 105 includes at leasttwo Sensors 110, the differing detections may be used to determine thedistance between the Augmented Reality Device 105 and the Beacon Device120. For example, the parallax or disparity between the images detectedby each Sensor 110 can be used to calculate the distance to the BeaconDevice 120. In embodiments, the Augmented Reality Device 105 may utilizeonboard sensors including accelerometers, magnetometers, and the like,as well as the Sensors 110 detecting one or more Beacon Device(s) 120 inorder to accurately determine the location of the Augmented RealityDevice 105 in the physical space (and therefore, the user) in relationto the Beacon Device 120.

In the illustrated embodiment, the Interactive Device 125 is a robottoy. In some embodiments, the Interactive Device 125 may be a character(e.g., an action figure). In other embodiments, the Interactive Device125 is a toy such as a ball or a vehicle. Of course, the InteractiveDevice 125 may be any object capable of implementing embodiments of thepresent disclosure.

As illustrated, the Interactive Device 125 includes two Sensors 130. Inan embodiment, the Sensors 130 are cameras or other imaging sensors. Inother embodiments, the Sensors 130 may be other types of sensor, such asultrasonic sensors, infrared sensors, and the like. Although theillustrated embodiment includes two Sensors 130, in some embodiments,the Interactive Device 125 includes a single Sensor 130, or more thantwo Sensors 130. Similarly, although both Sensors 130 are illustrated asfacing the same direction, in some embodiments the Sensors 130 may facein different directions from each other, and need not occupy the “eyes”of the Interactive Device 125. In an embodiment, each Sensor 130 is adifferent type of sensor. Further, in some embodiments, additionalsensors may be included in or on the Interactive Device 125, such asaccelerometers, tilt sensors, magnetometers, and the like. Inembodiments, data from the Sensors 130 is used by the Interactive Device125 to determine its location in the physical scene.

In an embodiment, the Beacon Device 120 can also be used to help locatethe Interactive Device 125 in the physical space. For example, as above,the Beacon Device 120 may be detected by the Sensors 130 of theInteractive Device 125. For example, in the illustrated embodiment, theInteractive Device 125 utilizes the Sensors 130 to detect the BeaconDevice 120, as illustrated by lines 140. In an embodiment, based on thedetected light or signals from the Beacon Device 120, the InteractiveDevice 125 can determine its location in the physical space relative tothe Beacon Device 120.

For example, in an embodiment, the perceived location of the BeaconDevice 120 in the frame of the Sensors 130 can be used to determine thedirection and/or angle that the Augmented Reality Device 105 is facingat any given point in time, relative to the Beacon Device 120.Similarly, in an embodiment, the Beacon Device 120 can output differentlight or signals on different sides or faces, such that the InteractiveDevice 125 can determine where it is in relation to the Beacon Device120 based on the light that the Sensors 130 detect. In an embodiment,the detected intensity of the Beacon Device 120 can be used to determinethe distance the Interactive Device 125 is from the Beacon Device 120.

In some embodiments where the Interactive Device 125 includes at leasttwo Sensors 130, the differing detections can be used to determine thedistance between the Interactive Device 125 and the Beacon Device 120.For example, the parallax or disparity between the images detected byeach Sensor 130 can be used to calculate the distance to the BeaconDevice 120. In embodiments, the Interactive Device 125 may utilizeonboard sensors including accelerometers, magnetometers, and the like,as well as the Sensors 130 detecting one or more Beacon Device(s) 120 inorder to accurately determine the location of the Interactive Device 125in the physical space.

As discussed above, in the illustrated embodiment, both the InteractiveDevice 125 and the Augmented Reality Device 105 are configured todetermine their respective locations in the physical space. Of course,in some embodiments, multiple Interactive Devices 125 may be present inthe physical environment. Similarly, in some embodiments, multipleBeacon Devices 120 can be utilized in order to ensure accuratedeterminations of location.

In some embodiments, the Augmented Reality Device 105 cannot determinethe location of other physical objects, such as the Interactive Device125. As discussed above, however, the Interactive Device 125 candetermine its own location in various embodiments in much the same waythat the Augmented Reality Device 105 can determine its own physicallocation. Further, although the illustrated embodiment includes a BeaconDevice 120 to aid in the location detection, in some embodiments, theAugmented Reality Device 105 and the Interactive Device 125 candetermine their respective locations without the need for a BeaconDevice 120.

By establishing a communications pathway between the Augmented RealityDevice 105 and the Interactive Device 125, each device in the physicalspace can determine its own location, and share that location with allother devices in the physical space. Furthermore, by utilizing a BeaconDevice 120, each Augmented Reality Device 105 and Interactive Device 125may determine its location relative to the Beacon Device 120, andthereby determine its location relative to all other Augmented RealityDevices 105 and Interactive Devices 125. Of course, in embodimentswithout a Beacon Device 120, the Augmented Reality Device 105 and theInteractive Device 125 may be configured to determine their respectivelocations relative to some other object or location such that theirlocations with respect to each other can be determined.

Furthermore, if the location of any virtual elements is known to anydevice, each Augmented Reality Device 105 and Interactive Device 125 candetermine its location relative to each virtual element. For example, inan embodiment, the Augmented Reality Device 105 generates and maintainsthe virtual elements in order to display them to the user and therebycreate an immersive atmosphere where virtual features are included inthe physical space, creating an augmented scene. In such an embodiment,the Augmented Reality Device 105 may communicate the position of eachvirtual entity or element to the Interactive Device 125, so that theInteractive Device 125 can react and interact with the virtual elements.

FIG. 2 is a block diagram 200 of an Augmented Reality Device 205 and anInteractive Device 250, according to one embodiment disclosed herein. Inthe illustrated embodiment, the Augmented Reality Device 205 includes,without limitation, a Processor 210, Storage 215, Memory 220, I/ODevices 225, Sensor Device(s) 230, and a Network Interface 235.Generally, the Processor 210 retrieves and executes programminginstructions stored in the Memory 220. The processor 210 is included tobe representative of a single CPU, multiple CPUs, a single CPU havingmultiple processing cores, GPUs having multiple execution paths, and thelike. The Memory 220 is generally included to be representative of arandom access memory. The Network Interface 245 enables the AugmentedReality Device 205 to connect to a data communications network (e.g.,wired Ethernet connection or an 802.11 wireless network). Further, whilethe depicted embodiment illustrates the components of a particularAugmented Reality Device 205, one of ordinary skill in the art willrecognize that augmented reality devices may use a variety of differenthardware architectures. Moreover, it is explicitly contemplated thatembodiments of the invention may be implemented using any device orcomputer system capable of performing the functions described herein.

The I/O Devices 225 represent a wide variety of input and outputdevices, including displays, keyboards, touch screens, and so on. Forinstance, the I/O Devices 225 may include a display device used toprovide a user interface or display the augmented reality scene. As anexample, the display may provide a touch sensitive surface allowing theuser to select different applications and options within an application(e.g., to select and control a virtual object). Additionally, the I/ODevices 225 may include a set of buttons, switches or other physicaldevice mechanisms for controlling the Augmented Reality Device 205. Forexample, the I/O Devices 225 could include a set of directional buttonsused to control aspects of a video game played using the AugmentedReality Device 205.

The Memory 220 represents any memory sufficiently large to hold thenecessary programs and data structures. Memory 220 could be one or acombination of memory devices, including Random Access Memory,nonvolatile or backup memory (e.g., programmable or Flash memories,read-only memories, etc.). In addition, memory 220 and Storage 215 maybe considered to include memory physically located elsewhere; forexample, on another computer communicatively coupled to the AugmentedReality Device 205. Illustratively, the Memory 220 includes an ARApplication 240.

In an embodiment, the Augmented Reality Device 205 is a head-mounteddevice (such as Augmented Reality Device 105). As discussed above, in anembodiment, the Augmented Reality Device 205 utilizes Sensor Devices 230to track its position in a physical space. Utilizing this location andorientation information, the AR Application 240 can display virtualelements such that they appear to be in the physical space. In anembodiment, as the user moves around the space, the AR Application 240updates the display output such that the virtual elements appear toremain stationary in the augmented scene, as if they were real physicalobjects in the physical space. For example, if the user moves forward inthe physical space while facing a virtual element, the virtual elementmay grow larger such that it appears to remain in the same place in thephysical space while the user approaches it. Similarly, in anembodiment, the user can walk around the virtual elements to view themfrom all angles, while the virtual elements remain stationary relativeto the physical space.

In the illustrated embodiment, the AR Application 240 further createsand maintains the augmented scene that is displayed to the user. Forexample, in an embodiment, the AR Application 240 creates a virtualenvironment, and determines which virtual elements to include in thescene according to user input or predefined rules or features (e.g., aspart of a game). The AR Application 240 further establishes the locationof each virtual element based on the predefined rules, and monitors thelocation of each as they move around in the virtual environment createdby the AR Application 240. In an embodiment, the AR Application 240 canalso determine the corresponding location in the physical space for eachlocation in the virtual world, such that each virtual element can beassociated with a physical location as well.

In an embodiment, the user(s) can use various I/O Devices 225 tointeract with the virtual elements. For example, in some embodiments, ahandheld controller can be used to manipulate virtual elements as ifthey were real objects. Similarly, in an embodiment, one or more SensorDevices 230 track the location, orientation, and movement of the user'shands to allow the user to control virtual objects with his or her ownhands.

For example, if an augmented scene includes a virtual ball, the user maybe able to catch and throw the ball, kick it, be struck by it, and thelike. Of course, other interactions between the user and virtualelements are included within various embodiments. Non-limiting examplesinclude swinging a virtual sword, using a virtual bow and arrow, firinga virtual blaster, and the like. In an embodiment, in addition tovirtual objects, the virtual elements can include virtual charactersthat the user can interact with using the various I/O Devices 225.

In addition, in the illustrated embodiment, the Memory 220 additionallyincludes Dynamics 245. In an embodiment, the Dynamics 245 includerelationships and associations among the virtual elements in a scene andvarious inputs from the user. In an embodiment, the Dynamics 245 providecharacter traits of virtual elements in the augmented scene. Forexample, one Dynamic 245 may provide that a virtual character (e.g., avirtual hamster) is afraid of the user. Based on this Dynamic 245, ARApplication 240 can control the virtual character to move it around theaugmented scene (such that it appears to move around the physical space)to avoid or stay away from the user. Similarly, in some embodiments, theDynamics 245 includes relationships between virtual elements and theInteractive Device 250. For example, in an embodiment, a Dynamic 245 mayprovide that an Interactive Device 250 is afraid of (and shouldtherefore flee from) a virtual element.

The illustrated embodiment additionally includes an Interactive Device250. In an embodiment, the Interactive Device 250 is a physical object,such as a toy robot (e.g., Interactive Device 125), in the physicalspace. As discussed above, in an embodiment, the Interactive Device 250is configured to determine its own location and/or orientation in thephysical space by using the Sensor Devices 270. In the illustratedembodiment, the Interactive Device 250 includes, without limitation, aProcessor 255, Storage 260, Memory 265, Sensor Device(s) 270,Actuator(s) 275, and a Network Interface 280. Generally, the Processor255 retrieves and executes programming instructions stored in the Memory265. Processor 255 is included to be representative of a single CPU,multiple CPUs, a single CPU having multiple processing cores, GPUshaving multiple execution paths, and the like. The Memory 265 isgenerally included to be representative of a random access memory. TheNetwork Interface 280 enables the Interactive Device 250 to connect to adata communications network (e.g., wired Ethernet connection or an802.11 wireless network). Further, while the depicted embodimentillustrates the components of a particular Interactive Device 250, oneof ordinary skill in the art will recognize that augmented realitydevices may use a variety of different hardware architectures. Moreover,it is explicitly contemplated that embodiments of the invention may beimplemented using any device or computer system capable of performingthe functions described herein.

The Memory 265 represents any memory sufficiently large to hold thenecessary programs and data structures. Memory 265 could be one or acombination of memory devices, including Random Access Memory,nonvolatile or backup memory (e.g., programmable or Flash memories,read-only memories, etc.). In addition, Memory 265 and Storage 260 maybe considered to include memory physically located elsewhere; forexample, on another computer communicatively coupled to the InteractiveDevice 250. Illustratively, the Memory 265 includes an InteractionApplication 285. In embodiments, the Interaction Application 285generally receives location information from the Augmented RealityDevice 205 and controls the Actuator(s) 275 to cause the InteractiveDevice 250 to move in response to various virtual and/or physicalstimuli. Additionally, in some embodiments, the Interactive Device 250can include other output devices, such as audio (e.g., a speaker) andvisual (e.g., a display or lights) that are controlled by theInteraction Application 285.

In one embodiment, the Interactive Device 250 receives, via the NetworkInterface 280, information about the location of the Augmented RealityDevice(s) 205, which is used to determine the location of any users inthe physical space. Similarly, the Interactive Device 250 receives thelocation of each virtual element in the augmented scene. As discussedabove, the location of each virtual element is determined by the ARApplication 240 as the augmented scene is rendered and the virtualelements move and interact according to the predefined simulation, game,Dynamics 245, and the like. In some embodiments, this location data istransmitted frequently or continuously by the Augmented Reality Device205 via the Network Interface 235. That is, as soon as updated positioninformation (e.g., location, orientation, or both) is determined for theAugmented Reality Device 205 and any virtual elements in the augmentedscene, the data is transmitted to the Interactive Device 250.

In an embodiment, in addition to position information about each virtualelement, the Interactive Device 250 also receives an identification ofwhat each virtual element is, as well as associated characteristics ofeach virtual element. In an embodiment, the Interaction Application 285identifies any Dynamics 290 that are associated with or relevant to theidentified virtual elements. In one embodiment, the Dynamics 290 definethe relationships or character traits of the Interactive Device 250 thatare related to each virtual element. For example, one Dynamic 290 mayprovide that the Interactive Device 250 likes or dislikes a particularvirtual element. Similarly, in an embodiment, a Dynamic 290 may includeactions or reactions to specified virtual elements that the InteractiveDevice 250 should take.

For example, in one embodiment, upon determining the location of avirtual ball (e.g., by receiving data through the Network Interface280), the Interaction Application 285 may identify a Dynamic 290 relatedto the virtual ball that provides that the Interactive Device 250 shouldattempt to catch the virtual ball. Based on repeatedly updated positioninformation for the virtual ball, the Interactive Device 250 can thendetermine whether it should attempt to catch it (e.g., by utilizing theActuators 275 to raise an arm or appendage of the Interactive Device250), or if the Interactive Device 250 should pursue the virtual ball bymoving around the physical space, or if the Interactive Device 250should simply wait for the user to toss or kick the ball towards theInteractive Device 250.

In another embodiment, the Interaction Application 285 may identify aDynamic 290 related to a virtual element in the augmented scene thatcauses the Interactive Device 250 to move towards or away from thevirtual element. For example, the virtual element may be a virtual firethat the related Dynamic 290 instructs the Interactive Device 250 toavoid, or a virtual token that the Interactive Device 250 should movetowards and attempt to take, according to the related Dynamic 290. Ofcourse, any other Dynamics 290 are possible in a variety of embodiments.

Additionally, in some embodiments, the Interactive Device 250 transmitsits own position information to the Augmented Reality Device 205. Insuch an embodiment, each device shares its own location and orientationinformation repeatedly, such that each device can determine where theother device is located in relation to itself, as well as where eachvirtual element is located in relation to each physical device.

FIG. 3A illustrates an Augmented Reality Scene 300, according to oneembodiment of the present disclosure. The illustrated Scene 300 includesa Virtual Ball 305 and an Interactive Device 250. That is, asillustrated, the Virtual Ball 305 is a virtual element being renderedand displayed to a user by an augmented reality device (such asAugmented Reality Device 105 or 205). Similarly, Interactive Device 250is a real physical object in the physical space. In the illustratedembodiment, the Virtual Ball 305 is moving towards the InteractiveDevice 250. For example, as discussed above, the user may have thrown orkicked the Virtual Ball 305.

As discussed above, in the illustrated embodiment, the Augmented RealityDevice 205 updates the position of the Virtual Ball 305 as it movesthrough the virtual space, and repeatedly renders the updated augmentedscene. In this way, the Virtual Ball 305 appears to move across theaugmented scene, from the location occupied by the user towards thelocation occupied by the Interactive Device 250.

In FIG. 3B, the Virtual Ball 305 has moved closer to the InteractiveDevice 250. For example, as illustrated, the Virtual Ball 305 appearssmaller from the perspective of the Augmented Reality Device 205, as ifit is moving away. In the illustrated embodiment, the Interactive Device250 has identified one or more Dynamics 290 related to the Virtual Ball305, and has begun to respond accordingly. For example, as illustrated,the Interactive Device 250 has determined that in response to theVirtual Ball 305, the Actuators 275 should be activated to mimic anattempt to catch the Virtual Ball 305.

In an embodiment, several processes must be completed before interactionbetween the Interactive Device 250 and Virtual Ball 305 can beaccomplished based on this identified Dynamic 290. For example, TheInteractive Device 250 may first determine its own location, as well asthe location of the Virtual Ball 305. Additionally, in an embodiment,the Interactive Device 250 can determine the orientations of theInteractive Device 250 and the Virtual Ball 305. Furthermore, in someembodiments, the Interactive Device 250 determines the movement ofitself and of the Virtual Ball 305 (e.g., a directionality and speed atwhich each is moving).

As discussed above, in an embodiment, the Interactive Device 250 candetermine its own location by utilizing one or more Sensor Devices 270.For example, in one embodiment, the Interactive Device 250 determinesits location and orientation relative to a Beacon Device 120.Furthermore, in an embodiment, the Interactive Device 250 receivesposition information about the Virtual Ball 305 from the AugmentedReality Device 205. This positioning information may be relative to theBeacon Device 120, relative to the Augmented Reality Device 205, orrelative to some other space or object (e.g., relative to something elsein the physical space). Additionally, in some embodiments, theInteractive Device 250 further receives position information for theAugmented Reality Device 205.

In some embodiments, the Beacon Device 120 is utilized as an “origin”for the augmented scene. For example, if a Cartesian plane is used, eachlocation in the augmented scene can be defined as an (x, y) coordinate,with the beacon located at (0, 0). Of course, in some embodiments,three-dimensional space is relevant, and locations are defined using (x,y, z) coordinates and the beacon may be located at (0, 0, 0). Further,in some embodiments, the coordinates also include an orientation of thevirtual element or physical object. Similarly, some embodimentsadditionally include a movement vector for each object or virtualelement.

In an embodiment, once all required position information is determined,the Interactive Device 250 can determine whether the Virtual Ball 305 isclose enough that the Interactive Device 250 should attempt to catch it.Similarly, in some embodiments, the Dynamic(s) 290 may provide that ifthe Virtual Ball 305 is moving away from the Interactive Device 250 (oris moving towards the Interactive Device 250 but will miss theInteractive Device 250 because of poor aim), the Interactive Device 250should take some other action, such as emitting an audio effect orchasing after the Virtual Ball 305.

In the illustrated embodiment of FIG. 3B, the Interactive Device 250 hasdetermined that the Virtual Ball 305 is moving closer to the InteractiveDevice 250, and based on its movement vector, that the Virtual Ball 305will pass closely enough to attempt to catch it. In response, theInteraction Application 285 has activated one or more Actuators 275 tomove an arm of the Interactive Device 250 such that a hand or otherappendage is located at the location the Virtual Ball 305 is movingtowards. To accomplish this, in an embodiment, the Interactive Device250 must determine not only its own location, but also the location ofeach of its appendages given the activation state of each Actuator 275.

In FIG. 3C, the Interactive Device 250 has caught the Virtual Ball 305.As discussed above, in an embodiment, the Interactive Device 250 hasrepeatedly exchanged position information with the Augmented RealityDevice 205 in order to determine the updated location of the VirtualBall 305 as it approached the Interactive Device 250. In an embodiment,to determine whether the Virtual Ball 305 was caught or missed, thelocation of the appendage (e.g., its left arm) of the Interactive Device250 is compared with the location of the Virtual Ball 305. In someembodiments, the movement of the Virtual Ball 305 is also considered(e.g., if it was moving too quickly, the Interactive Device 250 mightmiss the ball regardless of whether it is in the correct location).Similarly, in some embodiments, the orientation of the InteractiveDevice 250 and/or the Virtual Ball 305 may be considered (e.g., if theInteractive Device 250 was facing the wrong way, or the orientation ofthe Virtual Ball 305 was relevant to whether it could be caught).

In some embodiments, the Interactive Device 250 determines whether theVirtual Ball 305 was caught, and transmits this information to theAugmented Reality Device 205. In other embodiments, the AugmentedReality Device 205 receives location information from the InteractiveDevice 250 and determines whether the Virtual Ball 305 has been caught.If the Virtual Ball 305 was caught, the Augmented Reality Device 205updates the augmented scene to display the Virtual Ball 305 such that itappears to be held by the Interactive Device 250 (as opposed tocontinuing to move away). Similarly, in an embodiment, as theInteractive Device 250 moves its left appendage (e.g., the one holdingthe Virtual Ball 305), the Augmented Reality Device 205 updates thelocation of the Virtual Ball 305 in the scene to match the locationprovided by the Interactive Device 250. Similarly, the display isupdated such that it appears that the Interactive Device 250 is holdingthe Virtual Ball 305.

In an embodiment, the Interactive Device 250 may determine, based on theDynamics 290, to throw the Virtual Ball 305 back towards the user. Insome embodiments, the actual movement of the Interactive Device 250 isused to determine the movement vector of the Virtual Ball 305 (e.g., howhard the Virtual Ball 305 is thrown and what direction it is headingin). Of course, in various embodiments, other Dynamics 290 may providevarious other actions the Interactive Device 305 should take in responseto the Virtual Ball 305.

In some embodiments, rather than utilizing the physical location of eachappendage of the Interactive Device 250, whether the Virtual Ball 305makes contact with (or is caught by) the Interactive Device 250 mayinstead simply depend on the determined location of the InteractiveDevice 250 itself. Of course, in various embodiments, other virtualelements, Interactive Devices 250, and Dynamics 290 may be involved.Further, in addition to the Interactive Device 250 interacting inresponse to the Virtual Ball 305, the Virtual Ball 305 (or other virtualelement) may react to the Interactive Device 250 (e.g., by bouncing offof the Interactive Device 250, by seeking or fleeing from theInteractive Device 250, and the like).

FIG. 4A illustrates an augmented reality Scene 400, according to oneembodiment of the present disclosure. As depicted, the Scene 400includes two Interactive Devices 250 a-b, and no virtual elements arecurrently being rendered. In the illustrated embodiment, the InteractiveDevice 250 a is a robot, and the Interactive Device 250 b is a wizard.Of course, in embodiments, the Interactive Device 250 need not becharacters, but may be any physical devices that can perform thefunctions described herein.

In FIG. 4B, a Virtual Emblem or Virtual Token 405 has been added to theScene 400. For example, the user may have added the Virtual Token 405using one or more I/O Devices 225, or it may have been addedautomatically via a game mechanic. In the illustrated embodiment, theInteractive Device 250 a has identified a Dynamic 290 providing that theInteractive Device 250 a does not like the Virtual Token 405. Further,in response to this identified Dynamic 290, Interactive Device 250 a hasdetermined that one or more Actuators 275 should be activated to causethe Interactive Device 250 a to frown.

Further, in the illustrated embodiment, Interactive Device 250 b hasidentified a Dynamic 290 which provides that the Interactive Device 250b likes the Virtual Token 405. In response to this identified Dynamic290, the Interactive Device 250 b has activated one or more Actuators275 to cause the Interactive Device 250 a to smile. Thus, in theillustrated embodiment, the Interactive Devices 250 a-b have differentDynamics 290, and may respond differently to the same virtual stimulus(e.g., Virtual Token 405).

In FIG. 4C, the Interactive Device 250 a has moved away from the VirtualToken 405, while the Interactive Device 250 b has moved towards theVirtual Token 405. As discussed above, each Interactive Device 250 a-bhas determined one or more physical movements to make in response to theVirtual Token 405 (based on the identified relevant Dynamics 290), andhas activated one or more Actuators 275 to achieve that movement.Because the Dynamic 290 identified by the Interactive Device 250 adefines a “fear” relationship with the Virtual Token 405, theInteractive Device 250 a has moved away from the Virtual Token 405. Asdiscussed above, in an embodiment, this involves determining thelocation of the Interactive Device 250 a, as well as the location of theVirtual Token 405.

Similarly, as illustrated, the Dynamic 290 identified by the InteractiveDevice 250 b defines an “attraction” relationship with the Virtual Token405, the Interactive Device 250 b has moved towards from the VirtualToken 405. As discussed above, in an embodiment, this involvesdetermining the location of the Interactive Device 250 b, as well as thelocation of the Virtual Token 405.

FIG. 5 is a flow chart illustrating a Workflow 500 for providinginteraction between physical devices and virtual elements, according toan embodiment disclosed herein. As illustrated, the Workflow 500involves processes performed by both an Augmented Reality Device 505(such as Augmented Reality Device 105 or 205) as well as by anInteractive Device 510 (such as Interactive Device 125 or 250). Asillustrated, the Augmented Reality Device 505 determines its ownphysical location and/or orientation at block 515, as is described inmore detail above. Similarly, at block 520, the Interactive Device 510determines its own physical location and/or orientation, as discussedabove. Additionally, the Augmented Reality Device 505 determines thelocation of any virtual entities/elements in the augmented scene atblock 525.

In the illustrated Workflow 500, at block 530, the Interactive Device510 transmits its physical location and/or orientation information tothe Augmented Reality Device 505. As discussed in some embodiments, thedetermined locations of the Augmented Reality Device 505 and virtualelements are also transmitted to the Interactive Device 510, and thedevices share location information with each other. At block 535, theAugmented Reality Device 505 identifies relevant dynamics. For example,the relevant dynamics may define interactions between the InteractiveDevice 510 and the virtual entities in the scene. Similarly, in theillustrated embodiment, the Augmented Reality Device 505 determines anaction that the Interactive Device 510 should undertake. In anembodiment, this determination is based on the identified dynamics andthe location of the devices and virtual elements in the scene.

In addition to the locations of each of the devices and virtualelements, in some embodiments, the determined action to be undertaken bythe Interactive Device 510 is based on the relative locations of each orthe distance between them. For example, in an embodiment where theInteractive Device 510 should dodge a virtual ball, the distance betweenthe Interactive Device 510 and the virtual ball may be relevant. If thevirtual ball is close to the Interactive Device 510 (or will passclosely), the determined physical action may involve a greater physicalmovement than if the virtual ball is further or will not pass asclosely.

Similarly, in some embodiments, the movement vectors of each device andvirtual element may be used to determine the action to be taken. In anembodiment, the movement vector includes the direction and speed atwhich the device or virtual element is moving. In an embodiment, if thevirtual ball is approaching more rapidly, the determined physical actionmay involve a more rapid or sudden movement of the Interactive Device510 than if the virtual ball was approaching more slowly. Additionally,in one embodiment, if the Interactive Device 510 is already moving, thedetermined action may be affected by its movement vector. For example,the determined action can be to accelerate, slow down, or changedirection. However, if the Interactive Device 510 is moving in a firstdirection, the determined action may involve relatively smalladjustments to direction, rather than completely reversing direction.

At block 540, Augmented Reality Device 505 transmits the determinedaction(s) to the Interactive Device 510, and the Interactive device 510performs the action at block 545. That is, in the illustratedembodiment, the Interactive Device provides its physical location toAugmented Reality Device 505, and is informed of what action it shouldtake based on the identified dynamics and the virtual elements. In anembodiment, the determined action can include physical movements, audioto play, visuals to display, and the like. For example, as above, thedetermined action may be to avoid the virtual entity (e.g., to move awayfrom a specified location in the physical space).

FIG. 6 is a flow chart illustrating a workflow 600 for providinginteraction between physical devices and virtual elements, according toan embodiment disclosed herein. As illustrated, the Workflow 600involves processes performed by both an Augmented Reality Device 505(such as Augmented Reality Device 105 or 205) as well as by anInteractive Device 510 (such as Interactive Device 125 or 250). Asillustrated, the Augmented Reality Device 505 determines its ownphysical location and/or orientation at block 615, as is described inmore detail above. Similarly, at block 620, the Interactive Device 510determines its own physical location and/or orientation, as discussedabove. Additionally, the Augmented Reality Device 505 determines thelocation of any virtual entities/elements in the augmented scene atblock 625.

At block 630, the Augmented Reality Device 505 transmits the location ofthe virtual entities or elements to the Interactive Device 510.Similarly, in some embodiments, the Augmented Reality Device 505transmits its own location to the Interactive Device 510. Additionally,in some embodiments, the Interactive Device 510 also transmits itsdetermined location to the Augmented Reality Device 505. At block 635,the Interactive Device identifies relevant dynamics, as discussed above.Finally, at block 640, the Interactive Device performs one or moreactions that were determined based on the identified dynamics and thelocation of each device and virtual entity. That is, in the illustratedembodiment, the Interactive Device 510 determines which actions to take,rather than being instructed by the Augmented Reality Device 505.

In embodiments of the present disclosure, the physical devices (e.g.,Augmented Reality Device 505 and Interactive Device 510) each determinetheir own physical locations and/or orientations, and share thisinformation with each other. By sharing this information, more immersiveinteractions can be achieved between the Interactive Device 510 and oneor more virtual elements that exist in an augmented scene provided bythe Augmented Reality Device 505. Rather than utilizing static objectsor objects that do not interact with the virtual elements, InteractiveDevices 510 can determine their own locations and move and interact withvirtual objects, without requiring the user to intervene. This allowsusers and Interactive Devices 510 to interact with the same virtualobjects seamlessly and in a deeply engaging manner.

As described herein, in an embodiment, the Augmented Reality Device 505generates the virtual elements, creates the augmented scene, and outputsthe augmented scene to the user. Similarly, in some embodiments, theAugmented Reality Device 505 receives input from the user to affect thevirtual elements, such as by moving them around the augmented scene.Conversely, in an embodiment, the Interactive Device 510 does notgenerate or create virtual elements or augmented scenes. Rather, in anembodiment, the Interactive Device 510 receives this information fromthe Augmented Reality Device 505, and reacts to the augmented scene andvirtual elements that the Augmented Reality Device 505 has created.

In addition to interacting with virtual elements and with the user, inone embodiment, Interactive Devices 510 may interact with each other aswell. In an embodiment, multiple Interactive Devices 510 also sharetheir location information with each other, in order to facilitateinteraction and immersion. For example, in one embodiment, two or moreInteractive Devices 510 may pass a virtual ball to each other. Ofcourse, in various embodiments, Interactive Devices 510 can interactwith each other via any virtual element or dynamic.

Additionally, in some embodiments, multiple Augmented Reality Devices505 (e.g., used by multiple users) can communicate with each other, andshare their respective physical locations in the physical space. In thisway, multiple users can interact with a single augmented scene. Forexample, in an embodiment, one of the Augmented Reality Devices 505 actsas the “master” device and generates and maintains the virtual elements,as discussed above. The location and orientation of each virtual elementmay be communicated with the other Augmented Reality Device 505, asdiscussed above. In this way, each of the two or more Augmented RealityDevices 505 can render the same virtual elements in the same locationsin the physical space, such that both users can interact with them. Ofcourse, there may also be one or more Interactive Devices 510 in thephysical space and interacting with the same virtual elements as theuser(s).

FIG. 7 is a flow chart illustrating a method 700 of providinginteractive physical devices, according to one embodiment of the presentdisclosure. The method 700 begins at block 705, where an AugmentedReality Device 105 determines, using detected output from a beacondevice, a physical location of the augmented reality device in aphysical space. At block 710, an Interactive Device 125 determines,using detected output from a beacon device, a physical location of theinteractive device in the physical space. The method 700 continues toblock 715, where the Augmented Reality Device 105 displays an augmentedreality scene, wherein the augmented reality scene includes the physicalspace and a first virtual element.

At block 720, a predefined dynamic is identified based oncharacteristics of the interactive device and the first virtual element.As discussed above, this identification may be achieved by the eitherthe Augmented Reality Device 105 or the Interactive Device 125. Themethod 700 proceeds to block 725, where a physical movement to performis determined based on the determined physical location of theinteractive device and the predefined dynamic. Again, as above, thisdetermination may be made by either the Augmented Reality Device 105 orthe Interactive Device 125 in various embodiments. Finally, at block730, the Interactive Device 125 activates one or more actuators to causethe determined physical movement.

FIG. 8 is a flow chart illustrating a method 800 of providinginteractive physical devices, according to one embodiment of the presentdisclosure. At block 805, an Interactive Device 125 determines,utilizing an output from a beacon device detected by the one or moresensor devices, a physical location of the interactive device in aphysical space. The method 800 then proceeds to block 810, where theInteractive Device 125 receives an indication that an augmented realityscene is being displayed, wherein the augmented reality scene includesthe physical space and a first virtual element. Further, at block 815,the Interactive Device 125 identifies a predefined dynamic based oncharacteristics of the interactive device and the first virtual element.The method 800 continues to block 820, where the Interactive Device 125determines a physical movement to perform based on the determinedphysical location of the interactive device and the predefined dynamic.Finally, at block 825, the Interactive Device 125 activates one or moreactuators to cause the determined physical movement.

In the preceding, reference is made to embodiments of the invention.However, it should be understood that the invention is not limited tospecific described embodiments. Instead, any combination of thefollowing features and elements, whether related to differentembodiments or not, is contemplated to implement and practice theinvention. Furthermore, although embodiments of the invention mayachieve advantages over other possible solutions and/or over the priorart, whether or not a particular advantage is achieved by a givenembodiment is not limiting of the invention. Thus, the followingaspects, features, embodiments and advantages are merely illustrativeand are not considered elements or limitations of the appended claimsexcept where explicitly recited in a claim(s). Likewise, reference to“the invention” shall not be construed as a generalization of anyinventive subject matter disclosed herein and shall not be considered tobe an element or limitation of the appended claims except whereexplicitly recited in a claim(s).

As will be appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system, method or computer programproduct. Accordingly, aspects of the present invention may take the formof an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

Computer program code for carrying out operations for aspects of thepresent invention may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Smalltalk, C++ or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects of the present invention are described below with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

Embodiments of the invention may be provided to end users through acloud computing infrastructure. Cloud computing generally refers to theprovision of scalable computing resources as a service over a network.More formally, cloud computing may be defined as a computing capabilitythat provides an abstraction between the computing resource and itsunderlying technical architecture (e.g., servers, storage, networks),enabling convenient, on-demand network access to a shared pool ofconfigurable computing resources that can be rapidly provisioned andreleased with minimal management effort or service provider interaction.Thus, cloud computing allows a user to access virtual computingresources (e.g., storage, data, applications, and even completevirtualized computing systems) in “the cloud,” without regard for theunderlying physical systems (or locations of those systems) used toprovide the computing resources.

Typically, cloud computing resources are provided to a user on apay-per-use basis, where users are charged only for the computingresources actually used (e.g. an amount of storage space consumed by auser or a number of virtualized systems instantiated by the user). Auser can access any of the resources that reside in the cloud at anytime, and from anywhere across the Internet. In context of the presentinvention, applications (e.g., AR Application 240) or related dataavailable in the cloud. For example, the AR Application 240 couldexecute on a computing system in the cloud and generate the augmentedscene. Doing so allows a user to access this information from anycomputing system attached to a network connected to the cloud (e.g., theInternet).

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder or out of order, depending upon the functionality involved. Itwill also be noted that each block of the block diagrams and/orflowchart illustration, and combinations of blocks in the block diagramsand/or flowchart illustration, can be implemented by special purposehardware-based systems that perform the specified functions or acts, orcombinations of special purpose hardware and computer instructions.

While the foregoing is directed to embodiments of the present invention,other and further embodiments of the invention may be devised withoutdeparting from the basic scope thereof, and the scope thereof isdetermined by the claims that follow.

What is claimed is:
 1. A system comprising: a beacon device; anaugmented reality device comprising: a first processor; and a firstmemory containing first computer program code that, when executed on thefirst processor, performs a first operation; and an interactive devicecomprising: a second processor; one or more actuators; and a secondmemory containing second computer program code that, when executed onthe second processor, performs a second operation; wherein the firstoperation performed by execution of the first computer program code ofthe augmented reality device comprises: determining, using detectedoutput from the beacon device, a physical location of the augmentedreality device in a physical space; and displaying an augmented realityscene, wherein the augmented reality scene includes the physical spaceand a first virtual element; wherein the second operation performed byexecution of the second computer program code of the interactive devicecomprises: determining, using detected output from the beacon device, aphysical location of the interactive device in the physical space;identifying a predefined dynamic based on characteristics of theinteractive device and the first virtual element; determining a physicalmovement to perform based on the determined physical location of theinteractive device and the predefined dynamic; and activating the one ormore actuators to cause the determined physical movement.
 2. The systemof claim 1, wherein determining the physical location of the augmentedreality device comprises determining the physical location of theaugmented reality device relative to the beacon device.
 3. The system ofclaim 1, wherein determining the physical location of the interactivedevice comprises determining the physical location of the interactivedevice relative to the beacon device.
 4. The system of claim 1, whereinthe second operation performed by execution of the second computerprogram code of the interactive device further comprises determining alocation of the first virtual element in the augmented reality scene. 5.The system of claim 1, wherein the second operation performed byexecution of the second computer program code of the interactive devicefurther comprises: determining an audio effect to output based on thedetermined physical location of the interactive device and thepredefined dynamic; and outputting the determined audio effect.
 6. Thesystem of claim 1, wherein the first operation performed by execution ofthe first computer program code of the augmented reality device furthercomprises updating the augmented reality scene such that the firstvirtual element moves in response to the physical movement of theinteractive device.
 7. The system of claim 1, wherein the physicalmovement of the interactive device comprises moving to a second physicallocation in the physical space.
 8. The system of claim 1, whereindetermining the physical movement is further based on a movement of thefirst virtual element.
 9. An interactive device comprising: a processor;one or more sensor devices; one or more actuators; a network interface;and a memory containing computer program code that, when executed on theprocessor, performs an operation comprising: determining, utilizing anoutput from a beacon device detected by the one or more sensor devices,a physical location of the interactive device in a physical space;receiving an indication that an augmented reality scene is beingdisplayed, wherein the augmented reality scene includes the physicalspace and a first virtual element; identifying a predefined dynamicbased on characteristics of the interactive device and the first virtualelement; determining a physical movement to perform based on thedetermined physical location of the interactive device and thepredefined dynamic; and activating the one or more actuators to causethe determined physical movement.
 10. The interactive device of claim 9,wherein determining the physical location of the interactive devicecomprises determining the physical location of the interactive devicerelative to the beacon device.
 11. The interactive device of claim 9,the operation further comprising determining a location of the firstvirtual element in the augmented reality scene.
 12. The interactivedevice of claim 9, the operation further comprising: determining anaudio effect to output based on the determined physical location of theinteractive device and the predefined dynamic; and outputting thedetermined audio effect.
 13. The interactive device of claim 9, whereinthe first virtual element moves in the augmented reality scene inresponse to the physical movement of the interactive device.
 14. Theinteractive device of claim 9, wherein the physical movement of theinteractive device comprises moving to a second physical location in thephysical space.
 15. The interactive device of claim 9, whereindetermining the physical movement is further based on a movement of thefirst virtual element.
 16. A method comprising: determining, utilizingan output from a beacon device detected using one or more sensors, aphysical location of an interactive device in a physical space;receiving an indication that an augmented reality scene is beingdisplayed, wherein the augmented reality scene includes the physicalspace and a first virtual element; identifying a predefined dynamicbased on characteristics of the interactive device and the first virtualelement; determining a physical movement to perform based on thedetermined physical location of the interactive device and thepredefined dynamic; and activating the one or more actuators to causethe determined physical movement.
 17. The method of claim 16, whereindetermining the physical location of the interactive device comprisesdetermining the physical location of the interactive device relative tothe beacon device.
 18. The method of claim 16, the method furthercomprising determining a location of the first virtual element in theaugmented reality scene.
 19. The method of claim 16, wherein the firstvirtual element moves in the augmented reality scene in response to thephysical movement of the interactive device.
 20. The method of claim 16,wherein determining the physical movement is further based on a movementof the first virtual element.